feat(module-source): expose createModuleSourcePasses() by boneskull · Pull Request #3159 · endojs/endo

boneskull · 2026-04-04T04:12:57Z

This exposes a new function, createModuleSourcePasses for use with @endo/parser-pipeline.

Changes to existing code should be considered a pure refactor (or at least that was the aim).

Refactored code for reuse between createModuleSourcePasses (see functor.js, source-options.js).
Fixes quite a few type problems, including incorrect types for SourceMapHook and SourceMapDetails.
Adds other types where things were obvious or involve public APIs

boneskull · 2026-04-04T04:12:59Z

Warning

This PR is part of a stack and targets branch boneskull/evasive-pipeline, not master.
DO NOT MERGE until feat(evasive-transform): expose createEvasiveTransformPass for use with @endo/parser-pipeline #3188 is merged into master.

📚 Pull Request Stack

branch: master
- fix(compartment-mapper,bundle-source): async parser support #3186 - branch: boneskull/async-parser
  - fix(module-source): fix SourceMapHook & other types #3187 - branch: boneskull/fix-module-source-types
    - feat(parser-pipeline): create @endo/parser-pipeline #3158 - branch: boneskull/ast-service
      - feat(evasive-transform): expose createEvasiveTransformPass for use with @endo/parser-pipeline #3188 - branch: boneskull/evasive-pipeline
        
        feat(module-source): expose createModuleSourcePasses() #3159 - branch: boneskull/module-source-pipeline (this PR)
        
        create Babel-based AST parser for CommonJS sources #3162 - branch: boneskull/cjs-ast-parser
        
        feat(compartment-mapper): expose findUnknownCanonicalNames() #3205 - branch: boneskull/export-find-canonical-names

Managed by gh-stack

changeset-bot · 2026-04-04T04:13:04Z

🦋 Changeset detected

Latest commit: eb2e3c0

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 8 packages

Name	Type
@endo/module-source	Minor
@endo/compartment-mapper	Patch
@endo/bundle-source	Patch
@endo/check-bundle	Patch
@endo/daemon	Patch
@endo/import-bundle	Patch
@endo/test262-runner	Patch
@endo/cli	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

Copilot

Pull request overview

This PR exposes a new public API, createModuleSourcePasses(), from @endo/module-source for consumption by @endo/parser-pipeline, while refactoring shared module-source logic (source-options + functor/record building) and tightening source map typing/flow between module-source, parser-pipeline, and compartment-mapper.

Changes:

Added createModuleSourcePasses() with analyzer/transform visitors + buildRecord() for parser-pipeline integration.
Refactored shared state + record construction into createSourceOptions() and buildFunctorSource() / buildModuleRecord().
Updated source map hook flow so parser-pipeline receives raw source map objects (with compartment-mapper wrapping/stringifying for the public hook), plus added/updated public types and tests.

Reviewed changes

Copilot reviewed 18 out of 20 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
yarn.lock	Adds Babel-related type packages to the lockfile.
packages/parser-pipeline/src/composed-parser.js	Switches sourceMapHook callsite to pass raw source map objects.
packages/module-source/tsconfig.json	Adjusts include globs and excludes `.d.ts` files.
packages/module-source/test/visitor-passes.test.js	Adds tests for the new `createModuleSourcePasses()` API.
packages/module-source/test/module-source.test.js	Removes now-unneeded `@ts-expect-error` on source map hook details.
packages/module-source/test/_benchmark-babel-plugin.js	Reuses shared `createSourceOptions()` for benchmarks.
packages/module-source/src/visitor-passes.js	Implements `createModuleSourcePasses()` API.
packages/module-source/src/types/visitor-passes.ts	Adds exported TypeScript types for the new API.
packages/module-source/src/types/module-source.ts	Adds exported types for module-source options + source maps.
packages/module-source/src/transform-source.js	Refactors visitor extraction + improves source map emission gating.
packages/module-source/src/transform-analyze.js	Refactors to shared helpers for options + record construction.
packages/module-source/src/source-options.js	Introduces shared mutable state bag creation + visitor extraction.
packages/module-source/src/module-source.js	Simplifies ModuleSource construction by copying from frozen record.
packages/module-source/src/functor.js	Centralizes functor source + record assembly/freezing.
packages/module-source/src/external.types.js	Adds runtime-empty module for type-only re-exports.
packages/module-source/src/external.types.d.ts	Re-exports public types from the package.
packages/module-source/src/babel-plugin.js	Adds type imports/docs and a couple TS suppression annotations.
packages/module-source/package.json	Adds Babel generator/traverse type deps.
packages/module-source/index.js	Re-exports public types + `createModuleSourcePasses()`.
packages/compartment-mapper/src/types/external.ts	Introduces `ParseSourceMapHook` and updates parse option typing.

Comments suppressed due to low confidence (1)

packages/module-source/src/transform-analyze.js:16

This file still imports the JSDoc type Options from ./module-source.js, but that typedef was removed in this PR in favor of ModuleSourceOptions from ./types/module-source.js. With checkJs enabled, this will cause a broken type import and downstream type errors. Update the import and the @param annotation to reference ModuleSourceOptions (or an equivalent exported type) instead.

/** @import {Options} from './module-source.js' */

const makeCreateStaticRecord = transformSource =>
  /**
   *
   * @param {string} moduleSource
   * @param {Options} options
   */

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Introduces a low-level analysis primitive `analyzeModule` which is exported from the `analyzer.js` subpath. - Adds `src/types/external.ts` as a single re-export index for the public types (`SourceMapHook`, `SourceMapHookDetails`, `SourceMapObject`, `ModuleSourceOptions`) and the new analyzer types. `src/external.types.d.ts` now delegates to it. - Adds `src/types/analyzer.ts` with `AnalysisContext<T>`, `ModuleAnalysisContext`, `AnalysisOptions`, and `VisitorPlugin` — typed by `tsc` rather than inferred from JSDoc. - Adds `src/shim.types.d.ts` for the global `ModuleSource` augmentation used by the SES shim. - `tsconfig.json` is tightened: `stripInternal` removes `@internal` declarations from emitted `.d.ts` files. - Update `typedoc.json` to also consider the additional entry points - We no longer need to provide custom `@babel/types` exports to these functions. So, `makeModulePlugins` and `makeCjsModulePlugins` no longer return `PluginFactory` functions (`({ types }) => ({ visitor })`). Both now return plain `{ analyzePlugin: { visitor }, transformPlugin: { visitor } }` objects with `@babel/types` imported at module scope. `transform-source.js` is updated accordingly. - Benchmark and test fixture updated for the new API shape. - Adds `./analyzer.js` as a new package export. The `analyzeModule(options?)` function creates a fresh per-parse context with `analyzePass` / `transformPass` (plain `Visitor` objects) and `buildRecord(code, location)`. The same function powers the `ModuleSource` constructor internally. This will be consumed by the forthcoming `@endo/parser-pipeline`.

This was referenced Apr 4, 2026

fix(compartment-mapper): handle async parsers #3160

Closed

feat(evasive-transform): expose createEvasiveTransformPass for use with @endo/parser-pipeline #3161

Merged

boneskull self-assigned this Apr 4, 2026

boneskull force-pushed the boneskull/ast-service branch from 62012e4 to 7fbfdd1 Compare April 4, 2026 04:25

boneskull force-pushed the boneskull/module-source-pipeline branch from c3239b5 to 0fb7919 Compare April 4, 2026 04:25

boneskull mentioned this pull request Apr 4, 2026

feat(parser-pipeline): create @endo/parser-pipeline #3158

Draft

boneskull added performance lavamoat bug Something isn't working labels Apr 4, 2026

boneskull force-pushed the boneskull/module-source-pipeline branch from 0fb7919 to 93e7e2c Compare April 4, 2026 23:26

boneskull mentioned this pull request Apr 5, 2026

create Babel-based AST parser for CommonJS sources #3162

Merged

boneskull force-pushed the boneskull/ast-service branch from 7fbfdd1 to f54da12 Compare April 5, 2026 01:08

boneskull requested a review from Copilot April 5, 2026 01:10

Copilot started reviewing on behalf of boneskull April 5, 2026 01:11 View session

Copilot AI reviewed Apr 5, 2026

View reviewed changes

Comment thread packages/module-source/src/types/module-source.ts Outdated

Comment thread packages/module-source/src/types/module-source.ts

Comment thread packages/module-source/src/types/module-source.ts Outdated

Comment thread packages/module-source/src/babel-plugin.js Outdated

boneskull force-pushed the boneskull/module-source-pipeline branch from 93e7e2c to 656bc39 Compare April 7, 2026 22:49

boneskull force-pushed the boneskull/ast-service branch from f54da12 to 126fef3 Compare April 7, 2026 22:57

boneskull force-pushed the boneskull/module-source-pipeline branch 2 times, most recently from 7594c39 to 84dd4ee Compare April 7, 2026 23:05

boneskull added the enhancement New feature or request label Apr 7, 2026

boneskull force-pushed the boneskull/module-source-pipeline branch from b1c50eb to 59ff80d Compare April 7, 2026 23:27

boneskull force-pushed the boneskull/ast-service branch from a9fdc61 to 9abf181 Compare April 7, 2026 23:33

boneskull force-pushed the boneskull/module-source-pipeline branch from 59ff80d to 9d57d5e Compare April 7, 2026 23:33

boneskull force-pushed the boneskull/ast-service branch from 9abf181 to 460895c Compare April 14, 2026 02:22

boneskull force-pushed the boneskull/module-source-pipeline branch 2 times, most recently from a7fa619 to bf7cc72 Compare April 14, 2026 03:10

boneskull force-pushed the boneskull/ast-service branch from 460895c to 1c799b4 Compare April 16, 2026 01:36

boneskull force-pushed the boneskull/module-source-pipeline branch from bf7cc72 to 4987029 Compare April 16, 2026 01:36

boneskull force-pushed the boneskull/evasive-pipeline branch from 123668c to e9ea11e Compare April 16, 2026 02:04

boneskull force-pushed the boneskull/module-source-pipeline branch from 51afef6 to 34801a3 Compare April 16, 2026 02:04

boneskull force-pushed the boneskull/evasive-pipeline branch from e9ea11e to ea0b89e Compare April 16, 2026 02:09

boneskull force-pushed the boneskull/module-source-pipeline branch from 34801a3 to 95389dd Compare April 16, 2026 02:09

boneskull force-pushed the boneskull/evasive-pipeline branch from ea0b89e to 44eb9f7 Compare April 16, 2026 02:13

boneskull force-pushed the boneskull/module-source-pipeline branch from 95389dd to 6d15d60 Compare April 16, 2026 02:13

boneskull force-pushed the boneskull/evasive-pipeline branch from 44eb9f7 to c75d6f4 Compare April 16, 2026 02:15

boneskull force-pushed the boneskull/module-source-pipeline branch from 6d15d60 to 5750f33 Compare April 16, 2026 02:15

boneskull force-pushed the boneskull/evasive-pipeline branch from c75d6f4 to 30884e4 Compare April 16, 2026 02:19

boneskull force-pushed the boneskull/module-source-pipeline branch from 5750f33 to 1fd899f Compare April 16, 2026 02:19

boneskull force-pushed the boneskull/evasive-pipeline branch from 30884e4 to 906447e Compare April 16, 2026 02:38

boneskull force-pushed the boneskull/module-source-pipeline branch from 1fd899f to f50448a Compare April 16, 2026 02:38

boneskull force-pushed the boneskull/evasive-pipeline branch from 906447e to 7d86046 Compare April 16, 2026 20:58

boneskull force-pushed the boneskull/module-source-pipeline branch from f50448a to 65e196f Compare April 16, 2026 20:58

boneskull force-pushed the boneskull/evasive-pipeline branch from 7d86046 to eaa80a6 Compare April 21, 2026 22:37

boneskull force-pushed the boneskull/module-source-pipeline branch from 65e196f to aefe20a Compare April 21, 2026 22:37

boneskull force-pushed the boneskull/evasive-pipeline branch from eaa80a6 to d23760a Compare April 22, 2026 04:14

boneskull force-pushed the boneskull/module-source-pipeline branch from aefe20a to 3073d30 Compare April 22, 2026 04:14

boneskull force-pushed the boneskull/evasive-pipeline branch from d23760a to 2a910b2 Compare April 23, 2026 01:42

boneskull force-pushed the boneskull/module-source-pipeline branch from 3073d30 to c130a92 Compare April 23, 2026 01:42

boneskull mentioned this pull request Apr 23, 2026

feat(compartment-mapper): expose findUnknownCanonicalNames() #3205

Merged

boneskull force-pushed the boneskull/evasive-pipeline branch from 2a910b2 to 0309977 Compare April 29, 2026 20:54

boneskull force-pushed the boneskull/module-source-pipeline branch from c130a92 to 2f6e9e5 Compare April 29, 2026 20:54

boneskull force-pushed the boneskull/evasive-pipeline branch from 0309977 to b72e412 Compare April 30, 2026 01:37

boneskull force-pushed the boneskull/module-source-pipeline branch from 2f6e9e5 to eb2e3c0 Compare April 30, 2026 01:37

Base automatically changed from boneskull/evasive-pipeline to boneskull/ast-service April 30, 2026 01:37

boneskull merged commit eb2e3c0 into boneskull/ast-service Apr 30, 2026
2 of 25 checks passed

boneskull deleted the boneskull/module-source-pipeline branch April 30, 2026 01:37

boneskull restored the boneskull/module-source-pipeline branch April 30, 2026 01:42

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(module-source): expose createModuleSourcePasses()#3159

feat(module-source): expose createModuleSourcePasses()#3159
boneskull merged 1 commit intoboneskull/ast-servicefrom
boneskull/module-source-pipeline

boneskull commented Apr 4, 2026

Uh oh!

boneskull commented Apr 4, 2026 •

edited

Loading

Uh oh!

changeset-bot Bot commented Apr 4, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

boneskull commented Apr 4, 2026

Uh oh!

boneskull commented Apr 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

📚 Pull Request Stack

Uh oh!

changeset-bot Bot commented Apr 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🦋 Changeset detected

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

boneskull commented Apr 4, 2026 •

edited

Loading

changeset-bot Bot commented Apr 4, 2026 •

edited

Loading